Systems and methods utilizing artificial intelligence for placental assessment and examination

ABSTRACT

Systems and methods for completing a morphological characterization of an image of a placenta and providing suggested pathological diagnoses are disclosed. A system includes programming instructions that, when executed, cause processing devices to execute commands according to the following logic modules: an Encoder module that receives the digital image of the placenta and outputs a pyramid of feature maps, a SegDecoder module that segments the pyramid of feature maps on a fetal side image and on a maternal side image, a Classification Subnet module that classifies the fetal side image and the maternal side image, and a convolutional IPDecoder module that localizes an umbilical cord insertion point of the placenta from the classified fetal side image and the classified maternal side image. The localized umbilical cord insertion point, segmentation maps for the classified fetal side and maternal side images are provided to an external device for determining the morphological characterization.

CROSS-REFERENCE TO RELATED APPLICATION

The present disclosure claims the benefit of priority to U.S.Provisional Application No. 62/888,838, filed Aug. 19, 2019 and entitled“AI-BASED PLACENTA ASSESSMENT AND EXAMINATION,” the entire contents ofwhich is incorporated herein in its entirety.

BACKGROUND Field

The present disclosure generally relates to image analysis systems andmethods, and more specifically, to systems and methods that analyzeimages of placentas using artificial intelligence to assess and examinethe placentas.

Technical Background

The placenta is a window into the events of a pregnancy and the healthof the mother and baby. However, a very small percentage of placentasaround the world are ever examined by a pathologist. Even in developedcountries like the U.S., placentas are examined and characterized by apathologist only when it is considered necessary and resources areavailable. Full pathological examination is expensive and timeconsuming. Pathologists or pathologist assistants perform a macroscopicor gross examination and select sections for microscopic examination.After processing, they examine sections under a microscope and produce awritten report that contains various measurements (e.g., the weight, thedisc diameter) and diagnoses (e.g., completeness or retained placenta,cord insertion type, shape category, meconium, chorioamnionitis, and/orthe like). In some specialty centers the gross examination may includephotography using specialized imaging equipment. These measurements andplacental diagnoses can be useful for both short-term and long-termclinical care of the mother and baby.

SUMMARY

In an aspect, a system for completing a morphological characterizationof a digital image of a placenta includes one or more processing devicesand one or more non-transitory, processor-readable storage mediumshaving programming instructions thereon that, when executed, cause theone or more processing devices to execute commands according to thefollowing logic modules: an Encoder module that receives the digitalimage of the placenta and outputs a pyramid of feature maps, aSegDecoder module that segments the pyramid of feature maps on a fetalside image and on a maternal side image, a Classification Subnet modulethat classifies the fetal side image and the maternal side image, and aconvolutional IPDecoder module that localizes an umbilical cordinsertion point of the placenta from the classified fetal side image andthe classified maternal side image. The localized umbilical cordinsertion point, a segmentation map for the classified fetal side image,and a segmentation map for the classified maternal side image areprovided to an external device for the purposes of determining themorphological characterization by the external device.

In another aspect, a system for providing a suggested pathologicaldiagnosis of a placenta based on image data pertaining to the placentaincludes one or more processing devices and one or more non-transitory,processor-readable storage mediums having programming instructionsthereon that, when executed, cause the one or more processing devices toreceive the image data pertaining to the placenta from a morphologicalcharacterization system, extract a first segmentation map for aclassified fetal side image of the placenta and a second segmentationmap for a classified maternal side image of the placenta from the imagedata, determine, from the first segmentation map and the secondsegmentation map, pixels pertaining to a target portion to obtain aprocessed placenta photo, transmit the processed placenta photo to aneural network together with a set of instructions for determining oneor more features of the target portion, receive an output from theneural network that comprises a determined pathological diagnosis fromthe one or more features of the target portion, and provide thedetermined pathological diagnosis to an external device as a suggestedpathological diagnosis of the placenta.

Additional features and advantages of the aspects described herein willbe set for the in the detailed description which follows, and in partwill be readily apparent to those skilled in the art from thatdescription or recognized by practicing the aspects described herein,including the detailed descript which follows, the claims, as well asthe appended drawings.

It is to be understood that both the foregoing general description andthe following detailed description describe various aspects and areintended to provide an overview or framework for understanding thenature and character of the claimed subject matter. The accompanyingdrawings are included to provide a further understanding of the variousaspects, and are incorporated into and constitute a part of thisspecification. The drawings illustrate the various aspects describedherein, and together with the description serve to explain theprinciples and operations of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments set forth in the drawings are illustrative and exemplaryin nature and not intended to limit the subject matter defined by theclaims. The following detailed description of the illustrativeembodiments can be understood when read in conjunction with thefollowing drawings, where like structure is indicated with likereference numerals and in which:

FIG. 1 schematically depicts an illustrative placental assessment systemaccording to one or more embodiments shown and described herein;

FIG. 2A depicts a block diagram of illustrative hardware componentslocated within a placental assessment device according to one or moreembodiments shown and described herein;

FIG. 2B depicts a block diagram of illustrative logic modules containedwithin one or more memory components of a placental assessment deviceaccording to one or more embodiments shown and described herein;

FIG. 3 schematically depicts a flow diagram of an architecture for amodel for morphological characterization of placentas according to oneor more embodiments shown and described herein;

FIG. 4 schematically depicts a flow diagram of an illustrative method ofinsertion-type categorization and related automated measurementsprocedures according to one or more embodiments shown and describedherein;

FIG. 5A depicts images of illustrative placental abruption according toone or more embodiments shown and described herein;

FIG. 5B depicts images of illustrative placental chorioamnionitisaccording to one or more embodiments shown and described herein;

FIG. 5C depicts images of illustrative meconium examples according toone or more embodiments shown and described herein;

FIG. 5D depicts images of illustrative regular and irregular placentalshapes according to one or more embodiments shown and described herein;

FIG. 5E depicts images of illustrative true knots on the umbilical cordaccording to one or more embodiments shown and described herein;

FIG. 6A depicts images of an illustrative hypercoiled cord and a normalcord according to one or more embodiments shown and described herein;

FIG. 6B schematically depicts coil counting from an extracted edge fromthe images depicted in FIG. 6A according to one or more embodimentsshown and described herein;

FIG. 7A depicts a pixel-wise prediction confusion matrix according toone or more embodiments shown and described herein;

FIG. 7B depicts a pixel-wise prediction confusion matrix of a U-Netapproach;

FIG. 7C depicts a pixel-wise prediction confusion matrix of a Segnetapproach;

FIG. 7D depicts images of illustrative examples of segmentationapproaches according to one or more embodiments shown and describedherein;

FIG. 8A depicts a fetal/maternal-side classification confusion matrixwithout shared encoder weights according to one or more embodimentsshown and described herein;

FIG. 8B depicts a fetal/maternal-side classification confusion matrixwith shared encoder weights according to one or more embodiments shownand described herein;

FIG. 9A depicts a plot of a quantitative evaluation of insertion pointlocalization with a percentage of correct keypoints according to one ormore embodiments shown and described herein;

FIG. 9B depicts images of qualitative examples of insertion point heatmap predictions according to one or more embodiments shown and describedherein;

FIG. 10A depicts a plot of a receiver operating characteristic (ROC)curve for a classification network according to one or more embodimentsshown and described herein;

FIG. 10B depicts images of qualitative examples of incomplete partlocalization predictions produced by a localization network according toone or more embodiments shown and described herein;

FIG. 11A depicts an illustrative confusion matrix for an insertion typecategorization according to one or more embodiments shown and describedherein;

FIG. 11B depicts a plot of an illustrative quantitative evaluation of anestimation on the distance from an insertion point to a nearest discmargin according to one or more embodiments shown and described herein;

FIG. 11C depicts images of qualitative examples of insertion point typecategorization according to one or more embodiments shown and describedherein;

FIG. 12A depicts a plot of illustrative receiver operatingcharacteristic curves for detecting meconium according to one or moreembodiments shown and described herein;

FIG. 12B depicts a plot of illustrative receiver operatingcharacteristic curves for detecting abruption according to one or moreembodiments shown and described herein;

FIG. 12C depicts a plot of illustrative receiver operatingcharacteristic curves for detecting chorioamnionitis according to one ormore embodiments shown and described herein;

FIG. 13A depicts a plot of a comparison of mean average accuracy (MAP)between different ratios of the probability to sample an image withfalse knot or no knot over the probability to sample an image with trueknot (41) under IoU threshold 0.25, 0.5, and 0.75 according to one ormore embodiments shown and described herein;

FIG. 13B depicts a plot of a comparison of mean average accuracy (MAP)between different ratios of the probability to sample an image withfalse knot over the probability to sample an image with no knot (R2)under IoU threshold 0.25, 0.5, and 0.75, assuming R1=2 (the best fromthe results of FIG. 13A) according to one or more embodiments shown anddescribed herein;

FIG. 13C depicts a plot of a comparison of receiver operatingcharacteristic (ROC) curves between using RGB only vs. RGB+MASK as aninput at IoU threshold 0.5 according to one or more embodiments shownand described herein; and

FIG. 13D depicts images of example detection results using RGB+MASK asinput and R1=2 and R2=1.0 with IoU values also indicated for eachexample according to one or more embodiments shown and described herein.

DETAILED DESCRIPTION

Reference will now be made in detail to various aspects of systems andmethods for analyzing placentas, examples of which are illustrated inthe accompanying drawings. Whenever possible, the same referencenumerals will be sued throughout the drawings to refer to the same orlike parts.

The present disclosure generally relates to systems and methods ofautomatically assessing placentas utilizing artificial intelligence toanalyze image data pertaining to the placentas. The systems and methodsdescribed herein generally include addressing morphologicalcharacterization, which includes the tasks of placental imagesegmentation, umbilical cord insertion point localization, andmaternal/fetal side classification. The systems and methods describedherein also utilize clinically meaningful feature analyses of placentas,which comprises detection of retained placenta (e.g., incompleteplacenta), umbilical cord knot, meconium, abruption, chorioamnionitis,and hypercoiled cord, and categorization of umbilical cord insertiontype. The systems and methods described herein curate a datasetincluding about 1,300 placenta images with hand-labeled pixel-levelsegmentation map, cord insertion point and other information extractedfrom the associated pathology reports. The systems and methods furtherutilize an AI-based Placental Assessment and Examination system(AI-PLAX), which is a two-stage photograph-based pipeline for fullyautomated analysis. In a first stage, a three encoder-decoderconvolutional neural network with a shared encoder is used to addressmorphological characterization tasks by employing a transfer-learningtraining strategy. In a second stage, distinct sub-models are employedto solve different feature analysis tasks by using both the photographand the output of the first stage. The effectiveness of the pipeline isevaluated by using the curated dataset as well as the pathology reportsin the medical record. Through extensive experiments, it is demonstratedherein that the systems and methods are able to produce accuratemorphological characterization and very promising performance onaforementioned feature analysis tasks, all of which may possess clinicalimpact and contribute to future pregnancy research.

Automated placental assessment based on photographic imaging canpotentially allow more placentas to be examined, reduce the number ofnormal placentas sent for full pathological examination, and providemore accurate and timely morphological and pathological measurements oranalyses. Typical photographs of the placentas capture the umbilicalcord inserting into the fetal side of the disc, as well as the maternalside appearance. The systems and methods described herein focus on afully automated system for placental assessment and examination.Specifically, such systems will be responsible for placentalsegmentation, umbilical insertion point localization, fetal/maternalside classification, and the prediction of a number of pathologicalindicators (e.g., gross abnormality). These indicators include retainedplacenta (e.g., incomplete placenta), umbilical cord knot, meconium,abruption, chorioamnionitis, hypercoiled cord, and umbilical cordinsertion type. Some pathological findings from placentas are strictlymicroscopic; however, many have gross (macroscopic) and microscopicfeatures, while some are only seen on gross exam. The latter areparticularly frequent in placental pathology. Thus, a focus of thepresent disclosure includes, but is not limited to, predictingmacroscopic pathological indicators.

Existing placental imaging research can be classified into two typesbased on the time the image is taken: pre-delivery and post-delivery.Because a photo for the placenta under visible light spectrum cannot becaptured prior to the delivery, pre-delivery placental imaging researchhas been focused on images obtained through other means, such as, forexample, Mill and ultrasound. Pre-delivery placental imaging researchfocuses on segmentation, which can be used as visual aids for doctors.

Post-delivery placental imaging research engages different methods andthus can be further categorized into two types: those using microscopicimages and those using macroscopic images of the placenta taken bycameras. While microscopic assessment is more established, it requiresequipment and personnel to make slides and microscopes andmicrophotography to make images. In contrast, camera-based imaging inthe second category only requires an ordinary camera or even a cameraphone, and thus has greater potential to be widely adopted. Currentmacroscopic placental assessment from photos focus on a specific aspectand involved human assessment as a part of the process. For example,some assessments include studying variations in disc surface shape andvascular network from placental photos to identify associations betweenthese factors and vascular pathologies and placental efficiency. Othersattempt to estimate the size and shape of placentas from photos andfound placenta size but not shape to have an association with the birthweight. Currently, there has not been an automated approach to analyzeplacenta photographs. Such an approach has the potential for widespreadadoption because today's smartphones have high-quality cameras as wellas highly capable CPU, GPU, and/or AI chips.

The systems and methods described herein present a two-stage pipelinefor automated placental assessment and examination using photos. In thefirst stage (Stage I), we take a transfer learning (TL) approach totackle the associated tasks of morphological characterization ratherthan employing an independent model for each task. Transfer learningpromises performance gain and robustness enhancement throughrepresentation sharing for closely related tasks. The use of transferlearning may be summarized into three categories: “same domain,different tasks”, “different domains, same task” and “different domains,different tasks”. The systems and methods described herein are closestto the “same domain, different tasks” category but is not an exactmatch. More precisely, our method should fall into a category describedas “similar/overlapped domains, different tasks” because the source andtarget domains have overlap but are not the same, as described ingreater detail herein. Specifically, we transfer the learnedrepresentation of the encoder from the segmentation task to the othertwo tasks, e.g. disc side classification and insertion pointlocalization. Our network architecture design takes inspiration from therecent deep learning advances on classification, image, and key pointlocalization. In particular, the design of our segmentation modulefollows the practice of concatenating feature maps in encoder withfeature maps in decoder, such as performed in the U-Net; and the designof our insertion point module follows the practice of regressing aGaussian heat map, rather than using the coordinate values, as theground truth, which has been shown to be successful in humankey-point/joint localization tasks. In some embodiments, intermediatesupervision may be important to improving localization accuracy. Such anidea is taken in our design by considering two heat map predictions inthe final loss—one from the final feature layer and one from theintermediate feature layer. In the second stage (Stage II), we employindependent models each tailored for an individual task for a fewimportant placental assessment tasks including but not limited todetection of retained placenta (e.g., incomplete placenta), umbilicalcord knot, meconium, abruption, chorioamnionitis, hypercoiled cord, andcategorization of umbilical cord insertion type.

We chose to pursue a two-stage pipeline based on the followingobservations, both of which make it difficult to build an end-to-endmodel for all tasks: (1) Almost all of our second-stage tasks only applyto either the fetal side or the maternal side of a placenta or only tothe disc/cord/ruler region; and (2) A relatively small fraction of allimages bears the abnormalities we attempt to detect for the tasks in thesecond stage, and the sets of images bearing different abnormalitiesoften have little overlap.

The first observation makes it natural for the second-stage tasks totake in the segmentation and disc-side predictions from the first stageto narrow down the region of interest and eliminate irrelevantinformation. Also, this means the input feature space for these tasks israther different from the first stage or other second-stage tasks, andit is difficult, if not impossible, to apply transfer learning here tolet those tasks benefit from the representations learnt from othertasks. In contrast, tasks in the first stage are more closely relatedand have larger overlapped input feature space. The second observationmakes it sometimes impractical to use the same training/testing set forall tasks. Each task may have its own training/testing set such that themodel will not be dominated by negative cases (e.g., withoutabnormalities).

We summarize the primary contributions as follows. We introduce a novelpipeline for comprehensive, automated placental assessment andexamination using photos. The design of the pipeline, which has twostages, takes the relationship and the similarity of the tasks intoconsideration. Specifically, we use transfer learning to boostperformance and robustness for closely related tasks with significantoverlapped input space in the first stage. In the second stage, we usethe first-stage predictions in separate models to address distincttasks: to determine if an image is relevant (through sideclassification) and to provide the region of interest (throughsegmentation). Our method is explainable by design and achieves highlypromising results. We believe isolating the models for irrelevant tasksand enforcing strong priors on the information flow between sub-modelsare critical under a limited label and robustness-prioritized setting,which is typical for medical image analysis. Such isolation is necessaryto reduce the possibility of learning signals/correlations that do nothold true for the general distribution but just happen to be the case inour collected data based on prior domain knowledge. Additionally,distinct sub-models in the second stage can be developed in parallel andcan be upgraded without worrying that it will affect performance forother tasks. Our use of transfer learning for the first-stage tasks canbe categorized into the “similar/overlapped domains, different tasks”type, which is novel and can be applied to other medical image analysisproblems. We curated a first-of-its-kind large-scale dataset withhand-labeled segmentation maps, umbilical cord insertion point locationand diagnoses extracted from the associated pathology reports. Thisdataset enabled us to develop our computational pipeline addressingautomated placental assessment and examination tasks. We believe thedataset will also be highly beneficial to future research on theplacenta and adverse prenatal and postpartum outcomes.

The term “segmentation map” as used herein generally refers to a mapfrom image data that shows how each pixel in the image data isassociated with a semantic category, such as the various semanticcategories described herein (e.g. disc, cord, ruler, background). Thatis, the segmentation maps described herein may show how each pixel inthe image data is associated with a disc, how each pixel in the imagedata is associated with an umbilical cord, how each pixel in the imagedata is associated with a ruler, and/or how each pixel in the image datais associated with background.

Turning now to FIG. 1, an illustrative system 100 of utilizingartificial intelligence for placental assessment and examination isdepicted. The system 100, includes, but is not limited to, a network 105that is generally configured to electronically connect one or moresystems, devices, and/or components. Illustrative examples of systems,devices, and/or components that may be electronically connected to thenetwork 105 include, but are not limited to, a server computing device110, an imaging device 120, a user computing device 140, and anartificial intelligence system 130. While FIG. 1 only depicts a singleserver computing device 110, a single imaging device 120, a single usercomputing device 140, and a single artificial intelligence system 130,the present disclosure is not limited to such. That is, the system mayinclude one or more server computing devices 110, one or more imagingdevices 120, one or more user computing devices 140, and/or one or moreartificial intelligence systems 130 in other embodiments.

The network 105 may include any network now known or later developed,including, but not limited to, a wide area network (WAN), such as theInternet, a local area network (LAN), a mobile communications network, apublic service telephone network (PSTN), a personal area network (PAN),a metropolitan area network (MAN), a virtual private network (VPN), orany combination thereof.

The server computing device 110 is generally a computing device thatcontains components for executing various processes, such as receivingdata, cataloging data, cross-referencing data, recording data, providingdata, generating data, executing image recognition processes, executingassessment processes, executing examination processes, hostingapplications, providing user interfaces, interacting with applicationslocated on other devices, and/or the like according to embodiments shownand described herein. That is, the server computing device 110 mayinclude at least one or more processing devices and a non-transitorymemory component, where the non-transitory memory component includesprogramming instructions that cause the one or more processing devicesto execute the various processes described herein. In some embodiments,server computing device 110 may include a data storage component that isused for storing data, such as the data described herein. In someembodiments, server computing device 110 may include networking hardwarethat is used for communicating with the various components of the system100. Additional details regarding the server computing device 100 willbe described herein with respect to FIGS. 2A-2B.

The imaging device 120 is not limited by this disclosure, and maygenerally be any device that captures images. In some embodiments, theimaging device 120 may have optical components for sensing and capturingimages in the visible spectrum. In other embodiments, the imaging device120 may be particularly configured to sense electromagnetic radiation(e.g., thermal radiation). Accordingly, the imaging device 120 maygenerally be a device particularly tuned or otherwise configured toobtain images in spectra where particular types of radiation is readilydetected, such as the visible spectrum and the infrared spectrum(including the far infrared and the near infrared spectrum). As such,one illustrative example of a device particularly tuned or otherwiseconfigured to obtain images in spectra where heat radiation includes,but is not limited to, an infrared camera. In some embodiments, theimaging device 120 may be a camera that is sensitive within a range ofwavelengths of about 0.38 micrometer (μm) to about 14 μm, includingabout 0.38 μm, about 0.45 μm, about 0.485 μm, about 0.5 μm, about 0.565μm, about 0.59 μm, about 0.625 μm, about 0.74 μm, about 1 μm, about 2μm, about 3 μm, about 4 μm, about 5 μm, about 6 μm, about 7 μm, about 8μm, about 9 μm, about 10 μm, about 11 μm, about 12 μm, about 13 μm,about 14 μm, or any value or range between any two of these values(including endpoints). In certain embodiments, the imaging device 120may be a multispectral camera. Illustrative examples of suitable devicesthat may be used for the imaging device 114 include, but are not limitedto, an IR-camera (Infrared-camera), NIR-camera (Near Infrared-camera), aVISNIR-camera (Visual Near Infrared-camera), a CCD camera (ChargedCoupled Device-camera), and a CMOS-camera (Complementary Metal OxideSemiconductor-camera).

In some embodiments, the imaging device 120 may have a monochrome imagesensor. In other embodiments, the imaging device 120 may have a colorimage sensor. In various embodiments, the imaging device 120 may includeone or more optical elements, such as lenses, filters, and/or the like.In some embodiments, the imaging device 120 may further be a deviceparticularly configured to provide signals and/or data corresponding tothe sensed electromagnetic radiation to the control component 120. Assuch, the imaging device 114 may be communicatively coupled to thecontrol component 120, as indicated by the dashed lines depicted in FIG.1 between the imaging device 114 and the control component 120. In someembodiments, the imaging device 114 may have a 3D depth image sensor.

In various embodiments, the imaging device 120 may be positioned tocapture placenta images, such as the images described herein. That is,the imaging device 120 may generally be positioned such that a field ofview of the imaging device 120 captures at least a portion of a surfacesupporting a placenta and/or other objects. In some embodiments, theimaging device 120 may be mounted to any stationary or moving apparatusthat provides the imaging device with the capability of imaging theplacenta as described herein. For example, the imaging device 120 may becoupled to an arm or other support (not shown) that allows the imagingdevice 120 to move about an axis A around the placenta such that theimaging device 12 can capture any angle of the placenta. In someembodiments, movement of the imaging device 120 may be controlled (e.g.,remote controlled) by a user.

The user device 140 may generally provide an interface between a userand the other components connected to the network 105, including otherusers and/or other user computing devices. Thus, the user device 140 maybe used to perform one or more user-facing functions, such as receivingone or more inputs from a user or providing information to the user. Theuser device 140 may also be used to input additional data into any datastorage components of the systems, devices, and/or components of thesystem 100. The user device 140 may also be used to perform one or moreof the processes described herein. In some embodiments, the user device140 may be used to supply one or more of a placenta image, assessmentinformation, and examination results using an output device, such as adisplay, one or more radios, and/or the like, as described in greaterdetail herein.

It should be understood that while the user device 140 is depicted as apersonal computing device, this is a nonlimiting example. Morespecifically, in some embodiments, any type of computing device (e.g.,mobile device, tablet computing device, personal computer, server, etc.)may be used for any of these components.

The artificial intelligence system 130 is generally one or morecomputing devices (e.g., a collection of computing devices) that containhardware and software programming for hosting and operating one or moreartificial intelligence algorithms. The one or more artificialintelligence algorithms may generally be trained on existing data insuch a way that, when new data is received (e.g., new image datapertaining to a placenta, as described herein), particularcharacteristics of the new data can be determined and provided. Forexample, the algorithms hosted and operated by the artificialintelligence system 130 may receive image data pertaining to a placenta,categorize one or more features based on the image data, assess theplacenta in the image based on the one or more categorized features,and/or the like, as described in greater detail herein.

While each of these computing devices is illustrated in FIG. 1 as asingle piece of hardware, this is also merely an example. Morespecifically, each of the components depicted in FIG. 1 may represent aplurality of computers, servers, databases, components, and/or the like.

Illustrative hardware components of the server computing device 110 isdepicted in FIG. 2A. While the hardware components of the servercomputer device 110 are shown and described, the present disclosure isnot limited to such. For example, similar hardware components may alsobe included within any one of the various other systems, devices, and/orcomponents of the system 100, including, but not limited to, the imagingdevice 120, the user computing device 140, and/or the artificialintelligence system 130.

A local interface 200 may interconnect the various components of theserver computing device 110. The local interface 200 may be formed fromany medium that is capable of transmitting a signal such as, forexample, conductive wires, conductive traces, optical waveguides, or thelike. Moreover, the local interface 200 may be formed from a combinationof mediums capable of transmitting signals. In one embodiment, the localinterface 200 includes a combination of conductive traces, conductivewires, connectors, and buses that cooperate to permit the transmissionof electrical data signals to components such as processors, memories,sensors, input devices, output devices, and communication devices.Accordingly, the local interface 200 may include a bus. Additionally, itis noted that the term “signal” means a waveform (e.g., electrical,optical, magnetic, mechanical or electromagnetic), such as DC, AC,sinusoidal-wave, triangular-wave, square-wave, vibration, and the like,capable of traveling through a medium. The local interface 200communicatively couples the various components of the server computingdevice 110.

One or more processing devices 202, such as a computer processing unit(CPU), may be the central processing unit(s) of the computing device,performing calculations and logic operations required to execute aprogram. Each of the one or more processing devices 202, alone or inconjunction with one or more of the other elements disclosed in FIG. 2A,is an illustrative processing device, computing device, processor, orcombination thereof, as such terms are used within this disclosure.Accordingly, each of the one or more processing devices 202 may be acontroller, an integrated circuit, a microchip, a computer, or any othercomputing device. The one or more processing devices 202 arecommunicatively coupled to the other components of the server computingdevice 110 by the local interface 200.

One or more memory components 204 configured as volatile and/ornonvolatile memory, such as read only memory (ROM) and random accessmemory (RAM; e.g., including SRAM, DRAM, and/or other types of RAM),flash memories, hard drives, secure digital (SD) memory, registers,compact discs (CD), digital versatile discs (DVD), Blu-Ray™ discs, orany non-transitory memory device capable of storing machine-readableinstructions may constitute illustrative memory devices (i.e.,non-transitory processor-readable storage media) that is accessible bythe one or more processing devices 202. Such memory components 204 mayinclude one or more programming instructions thereon that, when executedby the one or more processing devices 202, cause the one or moreprocessing devices 202 to complete various processes, such as theprocesses described herein. Depending on the particular embodiment,these non-transitory computer-readable mediums may reside within theserver computing device 110 and/or external to the server computingdevice 110. A machine-readable instruction set may include logic oralgorithm(s) written in any programming language of any generation(e.g., 1GL, 2GL, 3GL, 4GL, or 5GL) such as, for example, machinelanguage that may be directly executed by the one or more processingdevices 202, or assembly language, object-oriented programming (OOP),scripting languages, microcode, and/or the like that may be compiled orassembled into machine readable instructions and stored in thenon-transitory computer readable memory (e.g., the memory components204). Alternatively, a machine-readable instruction set may be writtenin a hardware description language (HDL), such as logic implemented viaeither a field-programmable gate array (FPGA) configuration or anapplication-specific integrated circuit (ASIC), or their equivalents.Accordingly, the functionality described herein may be implemented inany conventional computer programming language, as pre-programmedhardware elements, or as a combination of hardware and softwarecomponents.

In some embodiments, the program instructions contained on the one ormore memory components 204 may be embodied as a plurality of softwaremodules, where each module provides programming instructions forcompleting one or more tasks. For example, referring now to FIG. 2B, theone or more memory components 204 may contain one or more of operatinglogic 240, segmentation logic 242, insertion point localization logic244, classification logic 246, placenta detection logic 248, knotdetection logic 250, meconium detection logic 252, umbilical cordinsertion type categorization logic 254, and/or hypercoiled corddetection logic 256. The operating logic 240 may include an operatingsystem and/or other software for managing components of the servercomputing device 110. The segmentation logic 242 may include programminginstructions or the like for segmenting image data, as described ingreater detail herein. The insertion point localization logic 244 mayinclude programming instructions or the like for localizing an umbilicalcord insertion point, as described herein. The classification logic 246may include programming instructions or the like for maternal/fetal sideclassification, as described herein. The placenta detection logic 248may include programming instructions or the like for detecting aplacenta from image data, as described herein. The knot detection logic250 may include programming instructions or the like for detecting anumbilical cord knot from image data, as described herein. The meconiumdetection logic 252 may include programming instructions or the like fordetecting meconium from image data, as described herein. The umbilicalcord insertion type categorization logic 254 may include programminginstructions or the like for classifying an umbilical cord by insertiontype, as described herein. The hypercoiled cord detection logic 256 mayinclude programming instructions for determining that an umbilical cordis hypercoiled from image data, as described in greater detail herein.

The various logic modules described herein with respect to one or morememory components 204 of the server computing device 110 are merelyillustrative, and that other logic modules, including logic modules thatcombine the functionality of two or more of the modules describedhereinabove, may be used without departing from the scope of the presentapplication. Furthermore, various logic modules that are specific toother systems, devices, and/or components of the system 100 of FIG. 1are also contemplated.

Referring again to FIG. 2A, one or more data storage devices 206, whichmay each generally be a storage medium that is separate from the one ormore memory components 204, may contain a data repository for storingdata that is used for storing electronic data and/or the like relatingto various data generated, captured, and/or the like, as describedherein. The one or more data storage devices 206 may be any physicalstorage medium, including, but not limited to, a hard disk drive (HDD),memory, removable storage, and/or the like. While the one or more datastorage devices 206 are depicted as local devices, it should beunderstood that at least one of the one or more data storage devices 206may be a remote storage device, such as, for example, a server computingdevice or the like in some embodiments.

Illustrative data that may be contained within the one or more datastorage devices 206 may include, but is not limited to, image data 222,pixel segmentation data 224, point localization data 226, classificationdata 228, feature analysis data 230, and/or the like. The image data 222may include, for example, data generated as a result of imagingprocesses completed by the imaging device 120 (FIG. 1), data providedfrom external image repositories, and/or the like, as described herein.Still referring to FIG. 2A, the pixel segmentation data 224 maygenerally be data that is generated as a result of one or more pixelsegmentation processes as described herein. The point localization data226 may generally be data that is generated as the result of one or morepoint localization processes and/or data that is used for the purposesof executing one or more point localization processes, as describedherein. The classification data 228 may be data that is generated as aresult of one or more classification processes and/or data that is usedby one or more classification processes (e.g., reference data), asdescribed herein. The feature analysis data 230 may be data that isgenerated as a result of one or more feature analysis processes and/ordata that is used by one or more feature analysis processes (e.g.,reference data), as described herein.

The types of data described herein with respect to one or more datastorage devices 206 of the server computing device 110 are merelyillustrative, and that types of data may be used without departing fromthe scope of the present application. Furthermore, various types of datathat are specific to other systems, devices, and/or components of thesystem 100 of FIG. 1 are also contemplated, such as data that isspecific to the imaging device 120, data that is specific to the userdevice 140 (e.g., user related data), data that is specific to theartificial intelligence system 130 (e.g., data generated as a result of,or to facilitate the operation of one or more artificial intelligencealgorithms and/or one or more machine learning algorithms).

Network interface hardware 208 may generally provide the servercomputing device 110 with an ability to interface with one or moreexternal components of the network 105 (FIG. 1), including one or moredevices coupled to the network 205 via the Internet, an intranet, or thelike. Still referring to FIG. 2A, communication with external devicesmay occur using various communication ports (not shown). An illustrativecommunication port may be attached to a communications network, such asthe Internet, an intranet, a local network, a direct connection, and/orthe like.

Device interface hardware 210 may generally provide the server computingdevice 110 with an ability to interface with one or more imaging devices120, including a direct interface (e.g., not via network 105 depicted inFIG. 1). Still referring to FIG. 2A, communication with such componentsmay occur using various communication ports (not shown). An illustrativecommunication port may be attached to a communications network, such asthe Internet, an intranet, a local network, a direct connection, and/orthe like.

AI interface hardware 212 may generally provide the server computingdevice 110 with an ability to interface with the artificial intelligence(AI) system 130, including a direct interface (e.g., not via network 105depicted in FIG. 1). Still referring to FIG. 2A, communication with suchcomponents may occur using various communication ports (not shown). Anillustrative communication port may be attached to a communicationsnetwork, such as the Internet, an intranet, a local network, a directconnection, and/or the like.

User device interface hardware 214 may generally provide the servercomputing device 110 with an ability to interface with the userinterface computing device 140, including a direct interface (e.g., notvia network 105 depicted in FIG. 1). Still referring to FIG. 2A,communication with such components may occur using various communicationports (not shown). An illustrative communication port may be attached toa communications network, such as the Internet, an intranet, a localnetwork, a direct connection, and/or the like.

It should be understood that in some embodiments, the network interfacehardware 208, the device interface hardware 210, the AI interfacehardware 212, and/or the user device interface hardware 214 may becombined into a single device that allows for communications with othersystems, devices, and/or components, regardless of location of suchother systems, devices, and/or components.

It should be understood that the components illustrated in FIG. 2A aremerely illustrative and are not intended to limit the scope of thisdisclosure. More specifically, while the components in FIG. 2 areillustrated as residing within the server computing device 110, theseare nonlimiting examples. In some embodiments, one or more of thecomponents may reside external to server computing device 110.Similarly, one or more of the components may be embodied in othercomputing devices not specifically described herein.

The systems, devices, and/or components described herein with respect toFIGS. 1 and 2A-2B generally provide functionality for carrying out aplurality of processes for determining placenta and placenta relatedcharacteristics from image data, as described herein. The remainingdescription provided herein includes specific details with respect tosuch processes.

Dataset

We collected a dataset including 18,400 placenta photos as well as theassociated pathology reports written in natural English by thepathologist who originally examined the placenta, spanning the years of2016 to 2018. The photos and reports are from Northwestern MemorialHospital, a large urban academic medical center. The photos were takenby on-site pathologists and pathologist assistants using a camerainstalled on a fixed height arm against standardized blue background.Pathology classification is standardized, and the pathologists haveperinatal training and expertise. From the 18,400 placenta photos (ofabout 9,000 placentas), 1,370 photos were selected to be hand labeled.665 of the photos are fetal-side images, and 705 are maternal-sideimages. We developed a web-based tool to collect the following data: i)the pixel-wise segmentation maps, ii) the side-type label as fetal sideor maternal side, and iii) the cord insertion point (only for fetalside, visualized as a Gaussian heat map centered at the markedcoordinate) so that multiple trained labelers could annotate thisdataset concurrently. We also extract diagnoses from the pathologyreports. A complete list of diagnoses we extracted from the pathologyreports are listed in Appendix A. For those placentas being diagnosedwith being retained/incomplete the pixel-wise incomplete area wasannotated by a highly-trained pathologist who is a research member(J.A.G.). For true knot in the cord, trained research members placed abounding box around the knot with expert review as needed.

We divided the fully-labeled dataset into training and testing sets withthe ratio of 0.8:0.2. Because the insertion point can only be observedfrom the fetal side, we only use the 665 fetal-side images for insertionpoint prediction, with the same training-testing ratio asaforementioned.

Stage I: Morphological Characterization

The proposed model for morphological characterization 300, asillustrated in FIG. 3, includes an Encoder 310 for feature pyramidextraction, which is shared among all tasks, a fully convolutionalSegDecoder 340 for placenta image segmentation on both fetal-side andmaternal-side images, a Classification Subnet 350 for fetal-side andmaternal-side classification, and a fully convolutional IPDecoder 330for insertion point localization. In some embodiments, the Encoder 310,the SegDecoder 340, the Classification Submet 350 and the IPDecoder 330may each be embodied as logic modules contained within the one or morememory components 204 (FIG. 2).

Encoder as Feature Pyramid Extractor

The Encoder 310 takes a placenta image x (either the fetal side or thematernal side) as the input and then outputs a pyramid of feature maps{f₁, f₂, f₃, f₄, f₅}.

Depending on the tasks, all or part of the feature maps are used byfurther task modules. Specifically, SegDecoder 340 takes {f₁, f₂, f₃,f₄, f₅} as input; Classification Subnet 350 takes {f₅} as input; andIPDecoder 330 takes {f₃, f₄, f₅} as input. The Conv-1 and Conv-2 blocks(blocks 312 and 314, respectively) both include a Conv-BatchNorm-Relulayer. The difference, however, is that the Cony layer in the Conv-1block (block 312) has stride 1, while the Cony layer in Conv-2 block(block 314) has stride 2. The Res cony blocks (e.g., block 316, block318, and block 320) are residual blocks with two convolutional layerswith stride 2 and 1, respectively, and the same kernel size 3×3, each ofwhich spatially downsamples the input feature maps to half of its sizeand doubles the number of feature channels. The residual structure ishelpful for training deep architectures.

SegDecoder for Segmentation

Our SegDecoder module 340 includes four expanding fully convolutionalblocks, each of which takes the concatenation of a copy of thecorresponding feature map f_(i), i∈{1,2,3,4}, and transposes aconvoluted (up-scaling factor 2) output feature map of the last layer.Finally, we apply soft-max to predict the probability of pixel (i,j)being of class k, denoted as p(i,j,k). To overcome the problem of highlyimbalanced number of pixels for different categories, we use dice loss(block 342) instead of the common cross entropy loss. Since we have fourclasses, we adjust the dice loss to suit the 4-class scenario:

$\begin{matrix}{{L^{seg} = {1 - \frac{\sum\limits_{i,j}{\sum\limits_{k = 0}^{3}{{p( {i,j,k} )} \cdot {g( {i,j,k} )}}}}{{\sum\limits_{i,j}{\sum\limits_{k = 0}^{3}{p^{2}( {i,j,k} )}}} + {g^{2}( {i,j,k} )}}}},} & (1)\end{matrix}$where i,j run over the row and column indexes of an image, respectively;p(i,j,k) and g(i,j,k) denote the predicted probability of the pixel atlocation (i,j) and the 0/1 ground truth of that pixel belonging to classk, respectively.Classification Subnet for Fetal/Maternal Side Classification

Because the fetal/maternal side can be inferred from the “disc” regionof a placenta alone, we crop the full placenta image x by a rectangleincluding the region of disc and resize the cropped image topredetermined dimensions (e.g., 512×512 pixels) as the input to theEncoder 310, which we denote as xc. The cropping is based on the groundtruth segmentation map during training and on the predicted segmentationmap at inference. Our Classification Subnet 350 includes a Res conyblock (block 322), two fully connected layers, and a soft-max layer. Atthe end, a binary cross entropy (BCE) loss is applied to supervise thenetwork at block 324.

IPDecoder for Insertion Point Localization

Because the insertion point is always located within or adjacent to the“disc” region, we use cropped disc region image x, just as we performcropping in Classification Subnet 350, as the input to the Encoder 310.Our IPDecoder 330 is also fully convolutional and includes two expandingfully convolutional blocks, the structure of which are the same as inthe first two convolutional blocks in SegDecoder 340. The similarity ofIPDecoder's 330 structure with SegDecoder's 340 helps us to ensure thatthe shared encoder representation could also be readily utilized here.Inspired by the success of intermediate supervision, we predict theinsertion point localization heat map after each expanding convolutionalblock by a convolutional layer with kernel size 1×1 (denoted as “Scoreblock” (block 332 and block 334) in FIG. 3) and use the mean squarederror (MSE) loss (block 336 and block 338) to measure the predictionerror:L ^(ip)=Σ_(i,j) ∥h(i,j)−ĥ(i,j)∥² ,k∈{1,2},  (2)where h(i,j) and ĥ(i,j) are the ground truth (Gaussian) heat map and thepredicted heat map, respectively. The final loss for insertion point isLip=Lip+Lip. During inference, the predicted insertion point location isdetermined by (i,j)=argmaxi,j ĥ(i,j).Training and Testing

We use mini-batched stochastic gradient descent (SGD) with learning rate0.1, momentum 0.9, and weight decay 0.0005 for all training. We use abatch size of 2 for all segmentation training and a batch size of 10 forall insertion point localization and fetal/maternal side classificationtraining. The procedures of training are as follows. We first train theSegDecoder 340+Encoder 310 from scratch with parameters initialized tozero. Next, we fix the learned weights for the Encoder 310 and trainClassification Subnet 350 and IPDecoder 330 subsequently (in otherwords, the Encoder only acts as a fixed feature pyramid extractor atthis stage). The rationale for making such choices is that the trainingfor segmentation task consumes all images we have gathered and makes useof pixel-wise dense supervision, which is much less likely to lead to anoverfitting problem. In contrast, the training for Classification Subnet350 takes binary value as ground truth for each image while the trainingfor IPDecoder 330 only uses around half of the whole dataset (onlyfetal-side images). To alleviate the lack of labels and to make themodel more robust, we use common augmentation techniques includingrandom rotation (±30°) as well as horizontal and vertical flipping forall training images.

Implementation

We implemented the proposed pipeline in PyTorch and ran experiments onan NVIDIA TITAN Xp GPU. For segmentation training, all images are firstresized to 768×1024, which is of the same aspect ratio as the originalplacenta images. For insertion point localization and fetal/maternalside classification training, we resize all cropped “disc” region imagesto 512×512, which is natural because the cropped “disc” regions oftenhave a bounding box close to a square. We summarize all parametersettings for our model in Appendix B.

Stage II: Placenta Feature Analysis

In this stage, we detect pathological indicators based on the resultsfrom Stage I.

Detection of Retained Placenta

Retained placenta is a cause of postpartum hemorrhage, and if prolonged,it can serve as a nidus for infection. Trained birth attendants performa focused examination of the placenta, including inspecting the maternalsurface for completeness. However, this process may fail if there is nota trained birth attendant, if blood obscures incomplete areas, or ifhuman error happens. Examination of placentas in pathology also includesassessment of the completeness of the maternal surface, which isrecorded in the pathology report. The treatment for retained placentaincludes removal of retained parts from the uterus. We identified 119out of 705 maternal side placenta images in our dataset with possible“retained placenta” based on the pathology reports and we asked aperinatal pathologist to annotate where the possible missing parts arefor each of the images. We trained two neural networks for this task,one for classification and one for localization.

The classification network is a binary classification convolutionalneural network (CNN) tasked with assessing if the placenta is retained(or incomplete) or not. As the incomplete parts are always within thedisk region, the pixels out of the disk region are not considered forthe binary classification and were excluded from the input. Thus, we usesegmentation maps predicted in Stage I to extract the disk part of aplacenta photo by setting pixels not classified as a part of the disc tozeros. Next, we feed the processed placenta photo into theclassification network, which is a Resnet-18 network, chosen to suit thesmall scale of our training set. In training, we fine-tune on ourdataset from a model pretrained on ImageNet (with 1,000 classes) usingmini-batched stochastic gradient descent (SGD) with batch size 10,learning rate 0.01, momentum 0.9, and weight decay 0.0005 for allexperiments.

The localization network assumes that the input placenta image has beenclassified as retained/incomplete and is tasked with segmenting out theretained/incomplete region(s). We treat it as a two-class segmentationproblem and train our localization network, which we choose to be theDeeplab architecture with ResNet-101 as the backbone network (pretrainedon ImageNet), against the expert-provided pixel-wise incomplete regionlabels. Segmentation map predicted in Stage I are used to excludenon-disc regions such that our localization network is not distracted bythose pixels. The training set contains 57 images and the testing setcontains 12 images. We use SGD with batch size 5, learning rate 0.01,momentum 0.9 and weight decay 0.0005.

Umbilical Cord Insertion Type Categorization

Abnormal cord insertion is a feature of fetal vascular mal-perfusion.Based on the segmentation, the predicted insertion point location, andthe scale we extracted from the ruler, we can measure the distance fromthe insertion point to the nearest margin of the disc, the length of thelong-axis and short-axis of the disc (all in centimeters). Further, weclassify the cord insertion type into “centrally”, “eccentrically”, and“marginally”, based on the ratio of the distance from the insertionpoint to its closest disc margin to the average length of the long-axisand short-axis. The thresholds for the above ratio between differentcategories are selected by optimizing classification accuracy on thetraining set. As illustrated in FIG. 4, the detailed procedures forinsertion type categorization and related automated measurements 400 areas follows.

-   -   1. We recover the occluded disc area by merging the originally        predicted disc area with the polygon defined by vertices        adjacent with both disc area and cord area at block 402. Here,        erosion and dilation image processing operations are used to        remove small holes sometimes appearing in the disc region given        by the raw segmentation prediction.    -   2. We extract the scale information from the ruler at block 404.        Since the ruler in the image could be of any orientation, we        extract the ruler region at step 406 and rectify the orientation        of the ruler and fit a rectangle from the predicted ruler region        at step 408. Next, we binarize the pixels within the ruler        region at step 410 such that the scale marker is more distinct.        Thirdly, we use kernel density estimation to fit a distribution        of the marker pixels (white after binarization) along the long        edge of the ruler at step 412. Finally, we read the number of        pixels corresponding to one centimeter as the number of pixels        between the two adjacent crests of the estimated distribution at        step 414.    -   3. We estimate the long-axis and short-axis of a placenta by        simulating how pathologists measure those from a 2-D shape by        using a vernier caliper at block 416.    -   4. We estimate the distance from the insertion point to its        nearest point on disc margin at block 418.    -   5. We calculate the ratio of the distance from the insertion        point to its closest disc margin to the average length of the        long-axis and short-axis and conduct the classification based on        pre-selected thresholds based on optimizing training set        classification accuracy at block 420.        Meconium, Abruption and Chorioamnionitis Detection

Meconium discharge is an indication of fetal distress and can damage theumbilical vessels as well as injure neonatal lungs. Meconium stains onthe fetal membranes and/or the fetal surface of the placenta are seen inFIG. 5A. Meconium is not always detectable from the gross colorexamination as shown in the third image (from left to right) of FIG. 5Aand histological analysis is required in some cases. Placental abruptionis separation of the placenta from the wall of the uterus before birthand can cause maternal blood loss and fetal distress or death. Atdelivery, dark red to brown adherent blood clots on the maternal side ofplacenta may be the main diagnostic characters of abruption; as seen inFIG. 5B, however, this complication is not always visible. Larger clotssuggest more severe abruption. Chorioamnionitis is an inflammation ofthe fetal membranes that often results from bacterial infection andwhich may progress to devastating infection of the fetus. The fetalsurface of the placenta that is affected by chorioamnionitis often looksopaque, with the color ranging from white to yellow to green. Thepercentage of placenta images diagnosed with meconium, abruption, orchorioamnionitis are relatively low. As a consequence, the number in ourfully labeled placenta images are too few for direct training of ourmodel. To address this challenge, we build our training and testing setfor these three tasks by using selected images of placentas diagnosedwith these three problems out of the 18,400 images we collected in theyear of 2016-2018. Specifically, we selected the set of images thatsatisfied our standards about freshness, non-placenta related objects inthe image, etc. In sum, we used 470 meconium diagnosed fetal side imagesfrom a total of 731 cases, 268 chorioamnionitis diagnosed fetal sideimages from a total of 461 cases, and 181 maternal side images withabruption diagnosis from a total of 314 cases. For each task, we buildthe training and testing set by 1) randomly sampling the same amount ofnegative cases (not diagnosed with meconium, abruption orchorioamnionitis) as positive cases as found in the whole dataset; 2)splitting the whole assembled dataset into training and testing setswith the ratio of 0.8:0.2.

We trained one simple 6-layer convolutional neural network as the binaryclassifier for each of the three abnormalities. Only the disc region ofan image is fed into those CNN classifiers and non-disc regions of theimage are zeroing out based on our segmentation predictions. The firstfour layers are convolutional layers with filter size of 3, stride of 1,max pooling (for downsampling), relu activation and output sizes are99×99×32, 48×48×64, 23×23×128, and 10×10×256, respectively. The last twolayers are fully connected layers with 1024 neurons and 1 neuron,respectively. At the end, a sigmoid activation is used to scale theoutput in the range of [0,1] as the probability for each class. We traineach network for 30 epochs (until which the training loss has converged)using RMSProp optimizer with learning rate 0.001, momentum 0.9, batchsize 10. Since abruption only appears on the maternal side andchorioamnionitis and meconium only appears on the fetal side, ourclassification network for each of them assumes a placenta image hasalready been classified into the associated side during inference.

Irregular Shape Detection

Abnormal placental shape has been associated with premature birth orstillbirth. The regular shape for a placenta is round or oval.Meanwhile, those placentas classified as irregularly shaped often looksstar-like or calabash-like (as shown in FIG. 5D), with prominent concaveor convex parts on the contour of the disc. By imitating how apathologist determines if the shape of a placenta disc is irregular, wedesign a simple measure to quantify the irregularity of the disc shapefor a placenta. First, we use the same module as in FIG. 4 to recoverthe occluded disc and produce a whole disc region as a binary map. Next,we find the best-fit ellipse using zeroth-, first-order and second-ordermoments. The (p+q)-th order moment is defined by:m _(p,q) =∫∫x ^(p) y ^(q) f(x,y)dxdy,  (3)where f(x,y)=1 when the pixel is on the disc area, and zero otherwise.Then we can get the center coordinates (x_(c),y_(c)), the inclinationangle α and the long-axis and short-axis a, b of the ellipse following:

$\begin{matrix}{{x_{c} = \frac{m_{1,0}}{m_{0,0}}},{y_{c} = \frac{m_{0,1}}{m_{0,0}}},} & (4) \\{{\alpha = {\frac{1}{2}{\tan^{- 1}( \frac{2m_{1,1}}{m_{2,0} - m_{0,2}} )}}},} & (5) \\{{a = \sqrt{\frac{2}{m_{0,0}}( {m_{2,0} + m_{0,2} + ( {( {m_{2,0} - m_{0,2}} )^{2} + {4m_{1,1}^{2}}} )^{\frac{1}{2}}} )}},} & (6) \\{{b = \sqrt{\frac{2}{m_{0,0}}( {m_{2,0} + m_{0,2} - ( {( {m_{2,0} - m_{0,2}} )^{2} + {4m_{1,1}^{2}}} )^{\frac{1}{2}}} )}},} & (7)\end{matrix}$

Finally, we count the number of pixels covered by the fitted ellipse(denoted as n1), the number of disc pixels outside the fitted ellipse(denoted as n2), and the number of non-disc pixels within the ellipse(denoted as n3, those pixels are white ones in FIGS. 5A-5E). We alsodefine

$\begin{matrix}{{I = \frac{n_{2} + n_{1}}{n_{1}}},} & (8)\end{matrix}$as the measure of irregularity for disc shape. Obviously, the larger theI, the more irregular a disc shape is. We select a threshold for I fromthe training set such that we classify a placenta as irregular-shaped ifits I is larger than that threshold. Two examples of regular andirregular shaped placentas, along with their disc binary maps and fittedellipses are displayed in FIG. 5D.Hypercoiled Cord Identification

As illustrated in FIG. 6A, a hypercoiled cord is more twisted than anormal cord, impairing fetal blood flow. Detecting this phenomenon isimportant because it is linked to infant mortality. Our approach is toapply Canny edge detection on the cord region predicted by oursegmentation model to detect fold crevices caused by hypercoiling. Thecount of those fold crevices could be a good approximation to the actualnumber of coils because it is also the main clue for pathologists toidentify an individual coil and count the total number of coils usingbare eyes. Before counting, we disregarded the detected fold crevicesthat are very small (in terms of length), crossed with the adjacent one,or whose orientation is too parallel with the orientation of the centralskeleton of the cord. FIG. 6A shows two examples of our intermediateresults for fold crevices detection. Sometimes, there are two or moreedge segments extracted for one crevice, which will result in incorrectcount of coils if we blindly count the number of extracted edgesegments. We design a simple but effective rule, as illustrated in FIG.6B, to overcome this:

-   -   Let e₁ ^(i), e₂ ^(i) and e₃ ^(i) be the points of intersection        between the i-th segment and the two cord boundaries and the        central skeleton, respectively. Let e₄ be e₂'s projection (in        the direction vertical to the central skeleton) onto the        opposite boundary.    -   Denote the length of the boundary between e₁ ^(i) and e₄ ^(i) as        T_(i). Denote the distance between e₁ ^(k) (k≥i+1) of the k-th        segment and e₁ ^(i) be d^(ik).    -   If d^(ik)>2T_(i), then the k-th segment will be counted as a        coil. Otherwise, the k-th segment will not be counted.

Let's denote n the count of coils we obtain following the above rule and1 the cord length in centimeters. We can quantify the coilness of a cordby:

$\begin{matrix}{C = \frac{n \times 10}{l}} & (9)\end{matrix}$e.g., the number of coils per ten centimeters. After exploring thehypercoiled cords in the training set, we define a cord to be“hypercoiled” if C≥4, which leads to the best training set accuracy whenit is used as the classification criterion.Knot Detection

A true knot forms when the umbilical cord ties around itself. FIG. 5Eshows some examples of true knots. Loosely looped knots do not usuallypresent a problem, while tight knots can severely affect the blood andoxygen exchange between the mother and the fetus. Therefore, knotdetection is included as a routine examination by clinical staff atdelivery and in the gross pathological exam. In a regular pathologyreport, a placenta is diagnosed with “normal” or “having true knots” or“having false knots” (which means the image does not contain trueknot(s) but some part(s) of cords are very similar to true knots), Inour dataset, we have 171 images diagnosed with having true knots and 462images diagnosed with having false knots. For each placenta imagediagnosed with having true knots, we manually labelled all the trueknots with bounding boxes. Using these labeled images, we trained ourknots detection module from scratch. For the knot detection task, we useYOLO, a single-stage detection network. We used the original RGB imageconcatenated with a binary mask denoting the cord region predicted byour segmentation as the input to the detection network and trained ourdetection network against the expert-labeled bounding boxes. As before,we used the 0.8:0.2 ratio to split the original dataset into trainingand testing sets. We used batch size 64 and learning rate 0.001.

RESULTS

In this section, we summarize the experimental results using ourdataset. The results are organized by the two stages and then by theindividual tasks within each stage. We also discuss the inference timeand the clinical significance at the end of this section.

Morphological Characterization

Segmentation

We compared our approach with two fully convolutional encoder-decoderarchitectures, the U-Net (Ronneberger et al., 2015) and the SegNet(Badrinarayanan et al., 2017). The results are shown in Table 1 belowand FIGS. 7A-7D.

TABLE 1 Segmentation evaluation Model pixel acc. class acc. mean IoUU-Net 98.10 92.98 88.21 SegNet 96.51 94.56 84.57 ours 98.73 97.26 93.93

We report the segmentation performance using standard segmentationmetrics pixel accuracy, mean accuracy, and mean IoU. The definition ofthose metrics are as follows: suppose we have counted how many pixelsare predicted to class j but with their ground truth being class i (forevery i,j∈{0, 1, . . . , k−1}, k is the number of classes) and we storeit as the term C_(i,j) in a k×k matrix C. We also denote the (groundtruth) total number of pixels for class i as T_(i). It is easy to seethat T_(i)=Σ_(j=0) ^(k-1)C_(i,j). The pixel accuracy, mean classaccuracy, and mean IoU are then defined as follows.

Pixel Accuracy:

$\begin{matrix}\frac{\sum\limits_{i = 0}^{k - 1}C_{i,j}}{\sum\limits_{i = 0}^{k - 1}T_{i}} & (10)\end{matrix}$

Mean Class Accuracy:

$\begin{matrix}{\frac{1}{k}\frac{\sum\limits_{i = 0}^{k - 1}C_{i,j}}{T_{i}}} & (11)\end{matrix}$

Mean IoU:

$\begin{matrix}{\frac{1}{k}{\sum\limits_{i = 0}^{k - 1}\frac{C_{i,j}}{T_{i} + {\sum\limits_{j \neq i}C_{i,j}}}}} & (12)\end{matrix}$

In FIGS. 7A-7C, we compare pixel-wise prediction confusion matrices ofour approach, U-Net, and Segnet, respectively, which reflects moredetails about segmentation performance for different categories. We alsoshow a few segmentation examples in FIG. 7D for qualitative comparison.Our approach yields the best segmentation results, especially fordifferentiating the cord and the ruler classes.

Fetal/Maternal Side Classification

We achieved an overall fetal/maternal side classification accuracy of97.51% on our test set. Without the shared encoder representation, wecan only achieve 95.52% by training Encoder+Classification Subnet fromscratch. We also compare their confusion matrices in FIGS. 8A-8B.

Insertion Point Localization

We choose Percentage of Correct Keypoints (PCK) as the evaluationmetric. PCK measures the percentage of the predictions fall within acircle of certain radius centered at the ground truth location. Moreformally, PCK at normalized distance x (x∈[0, 1]) is defined as:

$\begin{matrix}{{{{PCK}@x} = \frac{{{{p\text{:}\mspace{14mu}\frac{\sqrt{{{\overset{\hat{}}{p} - p}}_{2}}}{d}} < x} ⩓ {p \in \{ p_{i} \}_{i = 1}^{n}}}}{n}},} & (13)\end{matrix}$where {p_(i)}_(i=1) ^(n) are the n keypoints we are trying to predict.{circumflex over (p)} stands for our prediction for p; ∥⋅∥₂ stands forthe L-2 Euclidean distance and is used to measure the error of theprediction {circumflex over (p)} from the ground truth p; |⋅| stands forthe cardinality of a set. Herein, we choose the diameter of the disc asthe normalizing factor d. In practice, we approximate the diameter ofthe disc by the distance between the right most and left most pixel ofthe “disc” area in the segmentation map. In comparing our approach (bothwith and without shared encoder weights) to the Hourglass model (withnumber of stacks 1 and 2), we see competitive results achieved by ourapproach in human keypoint localization. FIG. 9A shows the PCK curves,with the x axis being the radius normalized by the diameter of theplacenta. Each curve in FIG. 9A is the average of the results for fivemodels trained with different seeds, and the light-colored band aroundeach curve (view-able when the figure is enlarged) shows the standarddeviation of the results. Our approach with shared Encoder 310 (FIG. 3)consistently gives the best results, especially when the normalizeddistance is from 0.2 to 0.6. We also show a few qualitative examples ofthe insertion point heat maps predicted by each model, along with theground truth in FIG. 9B.Placenta Feature Analysis

The predictions of the Stage I models enable us to conduct automaticplacenta feature analysis by subsequent models/procedures.

Detection of Retained Placenta

Both our classification network and localization network achievepromising results. We show the receiver operating characteristic curveof the classification network in FIG. 10A and example localizationresults along with the ground truth in FIG. 10B. To show the advantageof using the disc region only as the input, we compare two versions ofclassification network in FIG. 10A, one with segmented disc region only(with AUC 0.836) and one without using our segmentation predictions(with AUC 0.827). We also show the results of our classification networkbased on the disc regions provided by UNet (with AUC 0.781) and SegNet(with AUC 0.844) segmentation. The results based on our segmentationnetwork in Stage-I is significantly better than the results based onUNet, and on par with or slightly worse than the results based onSegNet. We have expanded our pool of images with expert-labeledincomplete region (around 2×) and improved our localization results fromIOU=0.571 to IOU=0.636 by training on this expanded pool of labeledimages. This improvement is also significant in our qualitative examplesshown in FIG. 10A.

Umbilical Cord Insertion Type Categorization

We achieved an overall 88% test accuracy and we show the classificationconfusion matrix in FIG. 11A. Because the ground truth distance from theinsertion point to its nearest point on the disc margin can be extractedfrom the pathology reports, as shown in Appendix A, we are able toevaluate our prediction for this important intermediate value. FIG. 11Bshows the evaluation for our estimation of the distance from theinsertion point to its nearest point on the disc margin on the test set.The x-axis represents the threshold of the normalized error (absoluteerror normalized by the ground truth) and the y-axis shows thepercentage of our estimation, the error of which is below suchthreshold. As shown, we have a 58% prediction accuracy if we set thethreshold to 0.2. Qualitative examples of our insertion typecategorization and associated automated categorization can be found inFIG. 11C. Insertion type predictions are displayed in the upper rightcorner of each image, along with the ground truth in brackets. Thesuccess cases are green boxed and the failed cases are red boxed. Foreach image, the predicted insertion point locations are marked with agreen dot; a transparent green mask is overlaid on the imagerepresenting the predicted whole disc region; a line is drawn betweenthe insertion point and its nearest point on the disc margin. Thepredicted length of such line is displayed next to it, along with theground truth length extracted from the pathology report (in brackets).The predicted long and short axes are also displayed, along with theirpredicted length in centimeters. We can see that the results for boththe umbilical cord insertion type categorization and its relatedmeasurements are very appealing. Our method is already very promising asa replacement for the current approach based on the manual measurementand naked-eye inspection.

Meconium, Abruption, and Chorioamnionitis Detection

The receiver operating characteristic (ROC) curves of binary classifiersfor meconium, abruption, and chorioamnionitis are shown in FIGS. 12A,12B, and 12C, respectively. We achieved 0.97/0.98, 0.72/0.72 and0.70/0.69 in terms of sensitivity and specificity for the detection ofabruption, meconium, and chorioamnionitis, respectively, under theselected operating point marked on the ROC curve as shown in FIGS.12A-12C. We also show ROC curves of binary classifiers for meconium,abruption, and chorioamnionitis based on UNet and SegNet segmentationsin each sub-figure. Overall, our segmentation network described inStage-I is the best choice to achieve the best ROC curve for all threetasks.

Irregular Shape Detection

In our dataset, 77 placentas are labeled as irregular shaped. Bymaximizing training set accuracy, we chose 0.14 as the irregularitymeasure (Eq. 8) threshold for classifying the shape. The sensitivity andspecificity for shape classification are 0.87 and 0.97, respectively,using the selected threshold. On expert review, the shape labels inpathology report are quite subjective, which we believe is the mainlimiting factor for achieving better classification performance in ourmodel. We can, however, make the shape classification much moreobjective by switching from the current naked-eye inspection approach toour computer-based approach.

Hypercoiled Cord Identification

Our dataset contains a total of 143 cords that are labeled ashypercoiled. The sensitivity and specificity for cord classification are0.85 and 0.93, respectively, under the selected coilness threshold. Webelieve the results still have room for improvement. The main factorshindering our method from achieving better accuracy include blood stainswithin the image, faint edges on the cord, limited number of hypercoiledcases for selecting the threshold, and the cord segmentation predictionerror.

Knot Detection

We used the standard metric, mean average precision (MAP) underdifferent thresholds of intersection over union (IoU) to evaluate ourdetection performance. In our dataset, the number of positive examplesis significantly less than the number of negative examples and thenumber of hard negative examples (false knot) is significantly less thanthe number of easy negative examples (no knot). Such imbalance ofdifferent classes and imbalance of easy cases and hard cases could hurtthe model's performance due to the dominating influence on the loss fromthe class in majority (or from the easy cases). This phenomenon has beenverified and studied in many other applications and models. To addresssuch a problem, we must balance the influence of different classes (oreasy/hard cases) on the loss, either through an explicit re-weightingscheme by multiplying a scalar or implicit re-weighting scheme byadjusting the sampling for SGD. In that regard, we explored differentsampling strategies instead of the default uniform sampling strategywhen we use SGD to train our detection network. We present the resultsin FIGS. 13A and 13B. Specifically, we swept the ratio of theprobability of sampling an image with a false knot or no knot over theprobability of sampling an image with a true knot (R1) and the ratio ofthe probability of sampling an image with a false knot over theprobability of sampling an image with no knot (R2). We then comparedetection performance on the same test set under the training settingswith different R1 (FIG. 13A) and different R2 (FIG. 13B). By default, ifwe sample uniformly from the training set, disregarding if a sample ispositive/negative or is a easy/hard case, R1=7 and R2=0.5. We can seefrom FIG. 13A and FIG. 13B that we can achieve significantly betterperformance by decreasing R2 and increasing R1 from the default value,which translates to forcing our model attend more to negative cases(false knot or no knot), especially the hard negative cases (falseknot). Under the best setting we selected (R1=2 and R2=1.0), we canachieve MAP 0.817, 0.813, 0.376 for IoU thresholds of 0.25, 0.5, and0.75, respectively. Given the detection results by our model, we areable to classify whether an image has a true knot. And sinceclassification itself is important in practice, we also show the ROCcurve for our model from a binary classification perspective in FIG.13C. As before, by concatenating the binary mask (given by oursegmentation model in stage 1) for the cord with the original image'sRGB channels, we achieve significant additional performance improvement.Quantitatively, we improved MAP from 0.77 to 0.81 and ROC curve from theblue line (AUC=0.89) to the orange line (AUC=0.93) in FIG. 13(c) byswitching from RGB only to RGB+Ours Mask as the input. Besides, when weconcatenate the segmented masks provided by UNet and SegNet instead ofthe segmentation network in Stage-I, the ROC curves become worse, andtheir AUC drop to 0.87 and 0.90, respectively. This again demonstratesthe superior performance of our segmentation method. A few qualitativeexamples of true knot detection (our best model with R1=2 and R2=1.0 andusing RGB+Ours MASK as input) are shown in FIG. 13D.

Inference Time and Discussion on the Clinical Significance

Inference Time

Table 2 below summarizes the inference time of each individual componentof our approach. For components not involving neural networks, weestimate the computation time by averaging over 10 images; forcomponents involving neural networks accelerated by GPU, we estimate thecomputation time by averaging the inference time for 20 batches ofimages. Inference batch size used for each component is also displayedin Table 2. If we conduct segmentation for the maternal and fetal imagesat the same time and all other steps sequentially, the total inferencetime for a placenta is about 3.26 second. Moreover, if we parallelizethe computation of Side classification and Insertion point estimation inStage-I and all parallelizable components in Stage-II, the totalinference time for a placenta is about 1.58 second. The inference timeof the bottleneck components for the total inference time estimation areunderlined in Table 2.

TABLE 2 Summary of inference time Component Inference time (s./img.)Batch size Segmentation 0.53  2 Side classification 0.09 10 Insertionpoint estimation 0.18 10 Retained placenta classification 0.11 32Retained placenta localization 0.47 10 Insertion type categorization0.87 NA Meconium detection 0.19 16 Abruption detection 0.23 16Chorioamnionitis detection 0.19 16 Irregular shape detection 0.39 NAHypercoiled cord identification 0.31 NA Knot detection 0.28 10Discussion on Clinical Significance

Our approach can significantly reduce the work burden of clinicians.Currently it takes about 15 minutes for a trained physician atNorthwestern Memorial Hospital to examine the placenta and produce apathology report that covers all diagnoses tackled by our approach,according to the perinatal pathologist (coauthor) in our team. This isabout 276 (569) times of the inference time of the sequential (parallel)version of our approach. More importantly, the benefits of a fullyautomatic system are not limited to faster inference time. Otherbenefits of our approach include:

-   -   High objectivity: There can be inconsistent diagnoses among a        group of physicians or even with the same physician over time.        Our approach, however, always predicts using the same set of        criteria and is a deterministic process.    -   24/7 availability and flexibility: For instance, if a woman        delivers on Saturday at noon, the placenta won't even make it to        the pathology lab until the next Monday morning. In contrast,        our approach can provide timely on-site diagnoses so prompt        treatment can be given to the mother and/or the baby if        necessary.    -   Scalability: By deploying our system in cloud services, we can        use more machines when the demand is high. In contrast, it's        costly to train and employ pathologists to meet sudden higher        demand of the service.

We proposed a two-stage pipeline to address the tasks for automatedplacental assessment and examination. In the first stage, we designed acompact multi-head encoder-decoder CNN to jointly solve morphologicalplacental characterization tasks by employing a transfer learningtraining strategy. We showed that our approach can achieve betterperformance than competitive baselines for each task. We also showedthat the representation learned from the segmentation task can benefitinsertion point localization and fetal/maternal side classificationtask. In the second stage, we used the output from the first stage, aswell as the original placenta photos, as the input and employed multipleindependent models for a few noteworthy placental assessment tasks.Through ablation experiments, we demonstrated that the predictions fromthe first stage models help us achieve better performance for tasks inthis stage. For second-stage placenta feature analysis tasks, though ourresults still have room to be improved, especially when more placentalimages diagnosed with those abnormalities are available in the future,our current approaches are already useful for triage purpose, whichcould significantly alleviate the workload for pathologists.

While particular embodiments have been illustrated and described herein,it should be understood that various other changes and modifications maybe made without departing from the spirit and scope of the claimedsubject matter. Moreover, although various aspects of the claimedsubject matter have been described herein, such aspects need not beutilized in combination. It is therefore intended that the appendedclaims cover all such changes and modifications that are within thescope of the claimed subject matter.

What is claimed is:
 1. A system for completing a morphologicalcharacterization of a digital image of a placenta, the systemcomprising: one or more processing devices; and one or morenon-transitory, processor-readable storage mediums having programminginstructions thereon that, when executed, cause the one or moreprocessing devices to execute commands according to the following logicmodules: an Encoder module that receives the digital image of theplacenta and outputs a pyramid of feature maps; a SegDecoder module thatsegments the pyramid of feature maps on a fetal side image and on amaternal side image; a Classification Subnet module that classifies thefetal side image and the maternal side image; and a convolutionalIPDecoder module that localizes an umbilical cord insertion point of theplacenta from the classified fetal side image and the classifiedmaternal side image, wherein the localized umbilical cord insertionpoint, a segmentation map for the classified fetal side image, and asegmentation map for the classified maternal side image are provided toan external device for the purposes of determining the morphologicalcharacterization by the external device.
 2. The system of claim 1,wherein the SegDecoder module includes a plurality of expanding fullyconvolutional blocks, wherein each of the plurality of expanding fullyconvolutional blocks transposes a convoluted output feature map of alast layer of a concatenation of a copy of the feature maps and appliessoft-max to predict a probability of a pixel being of a particularclass.
 3. The system of claim 2, wherein the SegDecoder module solvesthe following equation:${L^{seg} = {1 - \frac{\sum\limits_{i,j}{\sum\limits_{k = 0}^{3}{{p( {i,j,k} )} \cdot {g( {i,j,k} )}}}}{{\sum\limits_{i,j}{\sum\limits_{k = 0}^{3}{p^{2}( {i,j,k} )}}} + {g^{2}( {i,j,k} )}}}},$where i,j are coordinates of a location of the pixel, p(i,j,k) andg(i,j,k) denote a predicted probability of the pixel at the location(i,j) and a 0/1 ground truth of the pixel belonging to class k,respectively.
 4. The system of claim 1, wherein the digital imagereceived by the Encoder module is a full placenta image that is croppedto a region including a disc portion of the placenta and resized topredetermined dimensions.
 5. The system of claim 1, wherein theIPDecoder module comprises a plurality of expanding fully convolutionalblocks.
 6. The system of claim 5, wherein the IPDecoder module isconfigured to predict the insertion point as a localization heat mapafter each expanding convolutional block of the plurality of expandingfully convolutional blocks by a convolutional layer having a kernel sizeand using a mean squared error loss to measure a prediction error:L ^(ip)=Σ_(i,j) ∥h(i,j)−ĥ(i,j)∥² ,k∈{1,2}, where h(i,j) and ĥ(i,j) arethe ground truth (Gaussian) heat map and the predicted heat map,respectively.
 7. The system of claim 1, further comprising an imagingdevice communicatively coupled to the one or more processing devices,the imaging device capturing the digital image of the placenta.
 8. Thesystem of claim 1, further comprising an image repositorycommunicatively coupled to the one or more processing devices, the imagerepository storing the digital image of the placenta.
 9. The system ofclaim 1, wherein the morphological characterization provides theexternal device with an ability to predict a pathological diagnosis ofthe placenta, the pathological diagnosis selected from one or more of aretained placenta classification diagnosis, a retained placentalocalization diagnosis, an insertion type categorization diagnosis, ameconium detection diagnosis, an abruption detection diagnosis, achorioamnionitis detection diagnosis, an irregular shape detectiondiagnosis, a hypercoiled cord identification diagnosis, or a knotdetection diagnosis.
 10. A system for providing a suggested pathologicaldiagnosis of a placenta based on image data pertaining to the placenta,the system comprising: one or more processing devices; and one or morenon-transitory, processor-readable storage mediums having programminginstructions thereon that, when executed, cause the one or moreprocessing devices to: receive the image data pertaining to the placentafrom a morphological characterization system, extract a firstsegmentation map for a classified fetal side image of the placenta and asecond segmentation map for a classified maternal side image of theplacenta from the image data, determine, from the first segmentation mapand the second segmentation map, pixels pertaining to a target portionto obtain a processed placenta photo, transmit the processed placentaphoto to a neural network together with a set of instructions fordetermining one or more features of the target portion, receive anoutput from the neural network that comprises a determined pathologicaldiagnosis from the one or more features of the target portion, andprovide the determined pathological diagnosis to an external device as asuggested pathological diagnosis of the placenta.
 11. The system ofclaim 10, wherein the suggested pathological diagnosis is a retainedplacenta classification diagnosis, a retained placenta localizationdiagnosis, an insertion type categorization diagnosis, a meconiumdetection diagnosis, an abruption detection diagnosis, achorioamnionitis detection diagnosis, an irregular shape detectiondiagnosis, a hypercoiled cord identification diagnosis, or a knotdetection diagnosis.
 12. The system of claim 10, wherein the targetportion is one or more of the placenta, a disc, a ruler placed adjacentto the placenta, and an umbilical cord.
 13. The system of claim 10,wherein the neural network is a Resnet-18 network that is pretrained onImageNet using mini-batched stochastic gradient descent with batch size10, learning rate 0.01, momentum 0.9, and weight decay 0.0005.
 14. Thesystem of claim 10, wherein the neural network: recovers an occludeddisc area from the processed placenta photo by vertices adjacent with adisc area and a cord area; extracts scale information from a rulerwithin the processed placenta photo; binarizes a plurality of pixelswithin a region corresponding to the ruler to obtain a distinct scalemarker; uses kernel density estimation to fit a distribution of markerpixels from the distinct scale marker along a long edge of the ruler;reads a number of pixels corresponding to one centimeter as a number ofpixels between two adjacent crests of an estimated distribution;estimates a long-axis and a short-axis of the placenta; estimates adistance from an insertion point to a nearest point on a disc margin;calculates a ratio of the distance from the insertion point to thenearest point on the disc margin to an average length of the long-axisand the short-axis; and conducts a classification based on pre-selectedthresholds based on optimizing training set classification accuracy. 15.The system of claim 10, wherein the neural network is a 6-layerconvolutional neural network that is trained on a disc region of theplacenta to detect one or more of meconium, abruption, andchorioamnionitis in the placenta.
 16. The system of claim 10, whereinthe neural network: recovers an occluded disc area from the processedplacenta photo by vertices adjacent with a disc area and a cord area toproduce a whole disc region as a binary map; finds a best-fit ellipseusing zeroth-, first-order and second-order moments; counts a number ofpixels covered by the best-fit ellipse, a number of disc pixels outsidethe best-fit ellipse, and a number of non-disc pixels within thebest-fit ellipse; and use the number of pixels covered by the best-fitellipse, the number of disc pixels outside the best-fit ellipse, and thenumber of non-disc pixels within the best-fit ellipse as a measure ofirregularity for disc shape, wherein the neural network utilizes atraining set to classify the measure of irregularity as being irregularwhen above a predetermined threshold.
 17. The system of claim 10,wherein the neural network applies Canny edge detection to one or moreportions of the processed placenta photo indicated as a cord region todetect fold crevices caused by hypercoiling.
 18. The system of claim 10,wherein the neural network is a single edge detection network that usesthe processes placenta photo concatenated with a binary mask indicatinga cord region as an input trained against expert-labeled bounding boxesfor knots.